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Abstract 



We establish the existence of free energy limits for several combinatorial models on Erdos-Renyi 
graph G(2V, [cN\ ) and random r-regular graph G(N, r). For a variety of models, including indepen- 
dent sets, MAX-CUT, Coloring and K-SAT, we prove that the free energy both at a positive and 
zero temperature, appropriately rescaled, converges to a limit as the size of the underlying graph 
Ph diverges to infinity. In the zero temperature case, this is interpreted as the existence of the scaling 

Oh ' limit for the corresponding combinatorial optimization problem. For example, as a special case we 

prove that the size of a largest independent set in these graphs, normalized by the number of nodes 
converges to a limit w.h.p. This resolves an open problem which was proposed by Aldous |Ald-bj 
as one of his six favorite open problems. It was also mentioned as an open problem in several other 
places: Conjecture 2.20 in |Wor99j . [BR]. [JT08] and [AS03] 1. 

Our approach is based on extending and simplifying the interpolation method of Guerra and 
' Toninelli |GT02j and Franz and Leone |FL03] . Among other applications, this method was used to 

prove the existence of free energy limits for Viana-Bray and K-SAT models on Erdos-Renyi graphs. 
The case of zero temperature was treated by taking limits of positive temperature models. We provide 
instead a simpler combinatorial approach and work with the zero temperature case (optimization) 
C^l ' directly both in the case of Erdos-Renyi graph G(iV, |_ciVJ) and random regular graph G(N, r). In 

. addition we establish the large deviations principle for the satisfiability property of the constraint 

satisfaction problems Coloring, K-SAT and NAE-K-SAT for the G(7V, \ cN\ ) random graph model. 

on" 
o 

1 Introduction 

Consider two random graph models on nodes [N] = {1, . . . , N}, the Erdos-Renyi graph G(N, M) and the 
random r-regular graph G(N, r). The first model is obtained by generating M edges of the N(N — l)/2 
possible edges uniformly at random without replacement. Specifically, assume M = \cN\ where c > 
is a constant (does not grow with N). The second model G(N,r) is a graph chosen uniformly at 
random from the space of all r-regular graphs on N nodes, where the integer r is a fixed integer 
constant. Consider the size |X/v| of a largest independent set C [N] in G(N, [cN\) or G(N, r). It is 
straightforward to see that \In\ grows linearly with N. It was conjectured in several papers including 
Conjecture 2.20 in |Wor99j . |GJNS06j . [BR] , as well as [JT08] and [XS03] that \I N \/N converges in 
probability as N — > oo. Additionally, this problem was listed by D. Aldous as one of his six favorite 
open problems |Ald-b] . (For a new collection of Aldous' favorite open problems see [Ald-a] ) . The 
fact that the actual value of \In\ concentrates around its mean follows from a standard Azuma-type 
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inequality. However, a real challenge is to show that the expected value of |Xjv| normalized by N does 
not fluctuate for large N. 

This conjecture is in fact just one of a family of similar conjectures. Consider, for example, the 
random MAX-K-SAT problem - the problem of finding the largest number of satisfiable clauses of size 
K in a uniformly random instance of a K-SAT problem on N variables with cN clauses. This problem 
can be viewed as an optimization problem over a sparse random hypergraph. A straightforward argument 
shows that at least 1 — 2~ K fraction of the clauses can be satisfied with high probability (w.h.p.). It 
was conjectured in [CGHS04] that the proportion of the largest number of satisfiable clauses has a limit 
w.h.p. as N — > oo. As another example, consider the problem of partial (/-coloring of a graph: finding 
a g-coloring of nodes which maximizes the total number of properly colored edges. It is natural to 
conjecture again that value of this maximum has a scaling limit w.h.p. (though we are not aware of any 
papers explicitly stating this conjecture). 

Recently a powerful rigorous statistical physics method was introduced by Guerra and Toninelli |GT02j 
and further developed by Franz and Leone [FL03] . Franz, Leone and Toninelli [FLT03] . Panchenko and 
Talagrand [PT04] . and Montanari |Mon05] in the context of the theory of spin glasses. The method is 
based on an ingenious interpolation between a random hypergraph model on N nodes on the one hand, 
and a disjoint union of random hypergraph models on N% and N2 nodes, on the other hand, where 
N = N\ + N2. Using this method it is possible to show for certain spin glass models on random hyper- 
graphs, that when one considers the expected log-partition function, the derivative of the interpolation 
function has a definite sign at every value of the interpolation parameter. As a result the expected 
log-partition function of the A-node model is larger (or smaller depending on the details of the model) 
than the sum of the corresponding expected log-partition functions on N± and A^-node models. This 
super(sub)-additivity property is used to argue the existence of the (thermodynamic) limit of the ex- 
pected log-partition function scaled by N. From this property the existence of the scaling limits for the 
ground states (optimization problems described above) can also be shown by taking a limit as positive 
temperature approaches zero temperature. In [FL03] , the method was used to prove the scaling limit 
of log-partition functions corresponding to random K-SAT model for even K, and also for the so-called 
Viana-Bray models with random symmetric Hamiltonian functions. The case of odd K was also later 
resolved using the same method [FMj . 

Results and technical contributions. The goal of the present work is to simplify and extend 
the applicability of the interpolation method, and we do this in several important ways. First, we 
extend the interpolation method to a variety of models on Erdos-Renyi graphs not considered before. 
Specifically, we consider independent set, MAX-CUT, Ising model, graph coloring (henceforth referred 
to as Coloring), K-SAT and Not- All-Equal K-SAT (NAE-K-SAT) models. The coloring model, in 
particular, is of special interest as it is the first non-binary model to which interpolation method is 
applied. 

Second, we provide a simpler and a more combinatorial interpolation scheme as well as analysis. 
Moreover, we treat the zero temperature case (optimization problem) directly and separately from the 
case of the log-partition function, and again the analysis turns out to be substantially simpler. As 
a result, we prove the existence of the limit of the appropriately rescaled value of the optimization 
problems in these models, including independent set problem, thus resolving an open problem earlier 
stated. 

Third, we extend the above results to the case of random regular graphs (and hypergraph ensembles, 
depending on the model). The case of random regular graphs has been considered before by Franz, 
Leone and Toninelli [FLT03] for the K-SAT and Viana-Bray models with even number of variables per 
clause, and Montanari |Mon05] in the context of bounds on the performance of low density parity check 
(LDPC) codes. In fact, both papers consider general degree distribution models. The second of these 
papers introduces a multi-phase interpolation scheme. In this paper we consider a modification of the 
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interpolation scheme used in [FLT03| and apply it to the same six models we are focusing in the case 
of Erdos-Renyi graph. 

Finally, we prove the large deviation principle for the satisfiability property for Coloring, K-SAT 
and NAE-K-SAT models on Erdos-Renyi graph in the following sense. A well known satisfiability 
conjecture [Fri99 states that for each of these models there exists a (model dependent) critical value 
c* such that for every e > 0, when the number of edges (or clauses for a SAT-type problem) is at 
most (c* — e)N, the model is colorable (satisfiable) w.h.p. and when it is at least (c* + e)N, it is not 
colorable (not satisfiable) w.h.p. as JV -> oo. Friedgut [Fri99| came close to proving this conjecture 
by showing that these models exhibit sharp phase transition: there exists a sequence c* N such that for 
every e, the model is colorable (satisfiable) w.h.p. as ./V — > oo when the number of edges (clauses) is 
at most (c* N — e)N and is not colorable (satisfiable) w.h.p. when the number of edges (clauses) is at 
least (c* N + e)N. It is also reasonable to conjecture, and indeed was shown for the case K = 2, that not 
only the satisfiability conjecture is valid, but, moreover, the probability of satisfiability p(c,N) decays 
to zero exponentially fast when c > c*. 

In this paper we show that for these three models, namely Coloring, K-SAT and NAE-K-SAT, the 
limit r(c) = limTv^oo N~ l logp(c, N) exists for every c. Namely, while we do not prove the satisfiability 
conjecture and the exponential rate of convergence to zero of the satisfiability probability above the 
critical threshold, we do prove that if the convergence to zero occurs exponentially fast, it does so at 
a well-defined rate r(c). Assuming the validity of the satisfiability conjecture and the exponential rate 
of decay to zero above c* , our result implies that r(c) = when c < c* and r(c) < when c > c*. 
Moreover, we show that our results would imply the satisfiability conjecture, if one could strengthen 
Friedgut's result as follows: for every e > 0, p(c* N + e, N) converges to zero exponentially fast, where c* N 
is the same sequence as in Friedgut's theorem. 

Organization of the paper. The remainder of the paper is organized as follows. In the following 
section we introduce the sparse random (Erdos-Renyi) and random regular (hyper)-graphs and introduce 
various combinatorial models of interest. Our main results are stated in Section [3l The proofs for the 
case of Erdos-Renyi graphs are presented in Section [J] for results related to combinatorial optimization, 
and in Section[5]for results related to the log-partition function. The proofs of results for random regular 
graphs are presented in Section [H Several auxiliary technical results are established in the Appendix 
sections. In particular we state and prove a simple modification of a classical super- additivity theorem 
- if a sequence is nearly super-additive, it has a limit after an appropriate normalization. 

Notations. We close this section with several notational conventions. R(R + ) denotes the set of 
(non- negative) real values and Z(Z + ) denotes the set of (non- negative) integer values. The log function 
is assumed to be with a natural base. As before, [N] denotes the set of integers {1, . . . , N}. O(-) stands 
for standard order of magnitude notation. Specifically, given two positive functions f(N),g(N) defined 
on N <E Z + , / = 0(g) means sup N f(N)/g(N) < oo. Also / = o(g) means limjv->oo f(N)/g(N) = 0. 
Throughout the paper, we treat [N] as a set of nodes, and we consider splitting this into two sets of 
nodes, namely [N\] = {1, . . . , iVi} and {Ni + 1, . . . , N}. For symmetry, with some abuse of notation, 
it is convenient to denote the second set by [N2] where N2 = N — N±. A denotes the set-theoretic 
symmetric difference. Bi(iV, 0) denotes the binomial distribution with N trials and success probability 

6. Pois(c) denotes a Poisson distribution with parameter c, = stands for equality in distribution. A 
sequence of random variables Xn is said to converge to a random variable X with high probability 
(w.h.p.) if for every e > 0, limAr^oo P(| X^ — X\ > e) = 0. This is the usual convergence in probability. 
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2 Sparse random hypergraphs 



Given a set of nodes [N], and a positive integer K, a directed hyperedge is any ordered set of nodes 
(ii, ■ ■ ■ ,ik) ^ [-^1^- An undirected hyperedge is an unordered set of K not necessarily distinct nodes 
il,...,iK G [N]. A directed (undirected) if-uniform hypergraph on the node set [N] is a pair ([N], E), 
where E is any set of directed (undirected) -fT-hyperedges E = {e\, . . . , eigi}. A hypergraph is called 
simple if the nodes within each hyperedge e m ,l < m < \E\ are distinct and all the hyperedges are 
distinct. A (directed or undirected) hypergraph is called r-regular if each node i G [N] appears in 
exactly r hyperedges. The necessary condition for such a hypergraph to exist is Nr/K G Z+. A degree 
Aj = Aj(G) of a node i is the number of hyperedges containing i. A matching is a set of hyperedges 
such that each node belongs to exactly one hyperedge. In this paper we use the terms hypergraph and 
graph (hyperedge and edge) interchangeably. 

In order to address a variety of models in a unified way, we introduce two random directed hypergraph 
models, namely the Erdos-Renyi random graph model G(N, M), M G Z+ and the random regular graph 
&(N,r),r G Z + . These two graph models, each consisting of N nodes, are described as follows. The 
first G(N, M) is obtained by selecting M directed hyperedges uniformly at random with replacement 
from the space of all [N] hyperedges. A variant of this is the simple Erdos-Renyi graph also denoted 
for convenience by G(N,M), which is obtained by selecting M edges uniformly at random without 
replacement from the set of all undirected hyperedges each consisting of distinct K nodes. In this paper 
we will consider exclusively the case when M = [cN\ and c is a positive constant which does not grow 
with N. In this case the probability distribution of the degree of a typical node is Pois(afr) + 0(1/N). 
For this reason we will also call it a sparse random Erdos-Renyi graph. Often a sparse random Erdos- 
Renyi graph is defined by including each hyperedge in [N] K into the hypergraph with probability 
c/N K ~ l , and not including it with the remaining probability 1 — c/N K ~ l . The equivalence of two 
models is described using the notion of contiguity and and is well described in a variety of books, for 
example |AS92j . [JLRnU] . 

The second model G(N, r) is defined to be an r-regular directed iT-uniform hypergraph generated 
uniformly at random from the space of all such graphs. We assume Nr/K G Z+, so that the set of such 
graphs is non-empty. A simple (directed or undirected) version of G(N, r) is defined similarly. In this 
paper we consider exclusively the case when r is a constant (as a function of N) and we call G(N, r) a 
sparse random regular graph. 

From non-simple to simple graphs. While it is common to work with simple hypergraphs, for 
our purpose it is more convenient to establish results for directed non-simple hypergraphs first. It is 
well-known, however, that both G(N, M) and G(N, r) graphs are simple with probability which remains 
at least a constant as N — > oo, as long as c,r,K are constants. Since we prove statements which hold 
w.h.p., our results have immediate ramification for simple Erdos-Renyi and regular graphs. 

It will be useful to recall the so-called configuration method of constructing the random regular 
graph |Bol85] , |Bol80j , [Gal63j . Each node i is associated with r nodes denoted by j\, . . . ,j * We 
obtain a new set of Nr nodes. Consider the K- uniform matching e±, . . . ,e^/- r /K generated uniformly 
at random on this set of nodes. A JC-uniform matching is any set of -fT-hyperedges such that no two 
hyperedges have common nodes. From this set of edges we generate a graph on the original TV nodes 
by projecting each edge to its representative. Namely an edge ... ,ix) is created iff there is an 
edge of the form (j£ , . . . , j£ ) for some fci , . . . ,kx G [r] . The resulting graph is a random r-regular 
(not necessarily simple) graph, which we again denote by G(N, r). From now on when we talk about 
configuration graph, we have in mind the graph just described on Nr nodes. It is known [JLROO that 
with probability bounded away from zero as iV — >• oo the resulting graph is in fact simple. 

Given a hypergraph G = ([N], E) we will consider a variety of combinatorial structures on G, which 
can be defined in a unified way using the notion of a Markov Random Field (MRF). The MRF is a 
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hypergraph G together with an alphabet x = {0, 1) •••><? ~~ 1}) denoted by [q ], and a set of node and 
edge potentials Hi,i G [N],H e ,e G E. A node potential is a function Hi : [q~] — > R. and an edge 
potential is a function H e : [q~] K -)• {-00} U R. Given a MRF (G, [q~],Hi, H e , i G [iV] ,e £ E) and any 
x G for]* let 

= Hi( Xi ) + J2H e (x e ), H(G) = sup H(x), 
ie[N] eeE xe[q-] N 

where x e = (xi,i G e). Namely, -ff(x) is the value associated with a chosen assignment x and if is the 
optimal value, or the groundstate in the statistical physics terminology. In many cases the node and 
edge potentials will be random functions generated i.i.d. (see examples below). 

Associated with a MRF is the Gibbs probability measure on the set of node values [c/ - ]^ de- 
fined as follows. Fix a parameter A > assign the probability mass hg(x) = X h ^/Z<q to every 
assignment x G [q~] N , where Zg, = Yl x is the normalizing partition function. Observe that 

limA_>oo(log A) -1 log Zq, = H{G). Sometimes one considers A = exp(l/T) where T is temperature. The 
case T = 0, namely A = 00 then corresponds to the zero temperature regime, or equivalently the 
optimization (groundstate) problem. We distinguish this with a positive temperature case, namely 
A < 00. 

We will consider in this paper a variety of MRF defined on sparse random graphs G(N, \_cN\) and 
G(N,r). (In the statistical physics literature Xi are called spin values, and the corresponding MRF is 
called a diluted spin glass model.) We now describe some examples of concrete and well-known MRF 
and show that they fit the framework described above. 

Independent set. K = 2 and q = 2. Define H^l) = l,#i(0) = for all i G [N]. Define 
H e (l,l) = — 00, -ffe(l,0) = H e (0, 1) = H e (0, 0) = for every edge e = (n,^)- Then for every vector 
x G {0,1}* we have H(x) = —00 if there exists an edge ej = (11,12) such that x^ = Xi 2 = 1 and 
H(x) = \{i : Xi = 1}|, otherwise. Equivalently, H(x) takes finite value only on x corresponding to 
independent sets, and in this case it is the cardinality of the independent set. H(G) is the cardinality 
of a largest independent set. Note, that one can have many independent sets with cardinality H(G). 

MAX-CUT. K = 2 and q = 2. Define H^O) = H^l) = 0. Define H e (l,l) = H e (Q,0) = 
0,H e (l,0) = H e (0,l) = 1. Every vector x G {0,1}* partitions nodes into two subsets of nodes taking 
values and 1 respectively. H(x) is the number of edges between the two subsets. H(G) is the largest 
such number, also called maximum cut size. A more general case of this model is ^-coloring, see below. 

Anti-ferromagnetic Ising model. K = 2 and q = 2. Fix f3 > 0, B G R. Define Hi(0) = 
-B,Hi(l) = B. Define H e (l, 1) = H e (0,0) = -p,H e (l,Q) = H e (0, 1) = (5. It is more common to use 
alphabet { — 1, 1} instead of {0, 1} for this model. We use the latter for consistency with the remaining 
models. The parameter B, when it is non-zero represents the presence of an external magnetic field. 

(/-Coloring K = 2 and q is arbitrary. Hi(x) = 0,Vx G [q~] and H e (x,y) = if x = y and 
H e (x,y) = 1 otherwise. Therefore for every x G [q~] N , H(x) is the number of properly colored edges 
and H(G) is the maximum number of properly colored edges. 

Random K-SAT. K > 2 is arbitrary, q = 2. Hi = for all i G [N]. The edge potentials H e 
are defined as follows. For each edge e G E generate a e = (ai,... ,olk) uniformly at random from 
{0, 1}*, independently for all edges. For each edge e set H e (ai, . . . , ax) = and H e (x) = 1 for all other 
x = (x\, . . . , xk)- Then for every x G {0, 1}^, H(x) is the number of satisfied clauses (hyperedges) and 
H(G) is the largest number of satisfiable clauses. Often this model is called (random) MAX-K-SAT 
model. We drop the MAX prefix in the notation. 

NAE-K-SAT (Not-All-Equal-K-SAT). The setting is as above except now we set H e (a\, . . . , ax) = 
H e (l — ai, . . . ,1 — ax) = and H e (x) = 1 for all other x for each e. 
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It is for the K-SAT and NAE-K-SAT models that considering directed as opposed to undirected 
hypergraphs is convenient, as for these models the order of nodes in edges matters. For the remaining 
models, however, this is not the case. 

In several examples considered above we have had only two possible values for the edge potential H e 
and one value for the node potential. Specifically, for the cases of Coloring, K-SAT and NAE-K-SAT 
problems, H e took only values and 1. It makes sense to call instances of such problems "satisfiable" if 
H(G) = \E\, namely every edge potential takes value 1. In the combinatorial optimization terminology 
this corresponds to finding a proper coloring, a satisfying assignment and a NAE satisfying assignment, 
respectively. We let p(N,M) = F(H(G(N, M)) = M) denote the probability of satisfiability when 
the underlying graph is Erdos-Renyi graph G(N,M). We also let p(N,r) = P(H(G(N,r)) = rNK~ r ) 
denote the satisfiability probability for a random regular graph G(N,r). 

3 Main results 

We now state our main results. Our first set of results concerns Erdos-Renyi graph G(N, [cN\). 

Theorem 1. For every c > 0, and for every one of the six models described in Section^ there exists 
(model dependent) H(c) such that 

lim N~ 1 H(G(N, [cN\)) = H(c), (1) 

N—>oo 

w.h.p. Moreover, H(c) is a Lipschitz continuous function with Lipschitz constant 1. It is a non- 
decreasing function of c for MAX-CUT, Coloring, K-SAT and NAE-K-SAT models, and is a non- 
increasing function of c for the Independent set model. 
Also for every c > there exists p(c) such that 

lim A" 1 log p(N, \cN\ ) = p(c) , (2) 

N—t-oo 

for Coloring, K-SAT and NAE-K-SAT models. 

As a corollary one obtains the following variant of the satisfiability conjecture. 

Corollary 1. For Coloring, K-SAT and NAE-K-SAT models there exists a critical value c* H such that 
H(c) = c when c < c* H and H{c) < c when c > c* H . Similarly, there exists c*, such that p(c) = when 
c < c* and p(c) < when c > c*. 

Namely, there exists a threshold value c* such that if c < c* there exists w.h.p. as N — > oo a nearly 
satisfiable assignment (assignment satisfying all but o(N) clauses), and if c > c* then w.h.p. as N — > oo, 
every assignment violates linearly in N many clauses. The interpretation for Coloring is similar. The 
result above was established earlier by the second author for randomly generated linear programming 
problem, using the local weak convergence and martingale techniques [Gam04j. It would be interesting 
to see if the same result is obtainable using the interpolation method. 

Can one use Corollary [1] to prove the satisfiability conjecture in the precise sense? The answer would 
be affirmative, provided that a stronger version of Friedgut's result [Fri99| on the sharp thresholds for 
satisfiability properties holds. 

Conjecture 1. For the Coloring, K-SAT and NAE-K-SAT models there exists a sequence such that 
for every e > there exists 7 = 7(e) such that limAr^ 00 p(A, [(1 — e)M^J) = 1 andp(N, |_(l + e)M^J) = 
0(exp(- 7 A)), for all N . 
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In contrast, Friedgut's sharp phase transition result | Fri9 9] replaces the second part of this conjecture 
with (a weaker) statement lirnzv->oo p(N, [(1 + e)M^\) = 0. Thus, we conjecture that beyond the 
phase transition region M^, not only is the model not satisfiable w.h.p., but in fact the probability of 
satisfiability converges to zero exponentially fast. Conjecture [T] together with Theorem [1] implies the 
satisfiability conjecture. Indeed, it suffices to show that c* h is the satisfiability threshold. We already 
know that for every e > 0, p(N, [(l+ejc^N) — > 0, since H{(\ + e)c* h ) < (l + e)c* h . Now, for the other part 
it suffices to show that liminf/v M^/N — > c* h . Suppose not, namely there exists e > and a sequence 
N k such that (M^jN k ) + e < c* h for all k. Then (M^jN k ) + e/2 < c\ - e/2, implying that 

H(G(N k ,[M* Nk + (e/2)N k \)) 

M* Nk + (e/2)N k ~* [6) 

w.h.p. by Corollary[TJ But since grows at most linearly with N, then Mjy + (e/2)N k > (1 + e')M^ 
for some e' > for all k. By Conjecture Q] this means that p{N k , \_M^ + (e/2) A^.J ) — > exponentially 
in N k fast. But this means that there exists a sufficiently small 5 > such that deletion of every 
5N k edges (clauses) keeps the instance unsatisfiable w.h.p. Namely, H(G(N k , [M^ k + (e/2)iVfcJ)) < 
Mt + (e/2)N k - 5N k , w.h.p. as k — > oo, which contradicts ©. 

Let us now state our results for the existence of the scaling limit for the log-partition functions. 

Theorem 2. For every c > 0, A > 1, and for every one of the models described in Section^ there exists 
(model dependent) z(c) such that 

lim iV" 1 logZ(G(iV, LciVj)) = z(c), (4) 

N^oo 

w.h.p., where z{c) is a Lipschitz continuous function of c. Moreover, z(c) is non- decreasing for MAX- 
CUT, Coloring, K-SAT and NAE-K-SAT models, and is a non-increasing function of c for the Inde- 
pendent set model. 

Remark: The case A = 1 is actually not interesting as it corresponds to no interactions between the 
nodes leading to Z(G) = Uie[N] X^^i q -] H ^ X \ \ n this case the limit of A^" 1 log Z(G(N, [cN\)) exists 
trivially when node potentials Hi are i.i.d. For independent set, our proof holds for A < 1 as well. But, 
unfortunately our proof does not seem to extend to the case A < 1 in the other models. For the Ising 
model this corresponds to the ferromagnetic case and the existence of the limit was established in [DM 
using a local analysis technique. 

We now turn to our results on random regular graphs. 

Theorem 3. For every r G Z +; and for all of the models described in the previous section, there exists 
(model dependent) H(r) such that lnmy^^ jver _1 i<'z+ N^ 1 H(G(N, r)) = H(r) w.h.p. 

Note, that in the statement of the theorem we take limits along subsequence N such that NrK -1 
is an integer, so that the resulting random hypergraph is well-defined. Unlike the case of Erdos-Renyi 
graph, we were unable to prove the existence of the large deviation rate 

lim N~ 1 logp(N,r) 

N^oo,N£r~ 1 KZ + 

for the Coloring, K-SAT and NAE-K-SAT problems and leave those as open questions. 

Finally, we state our results for the log-partition function limits for random regular graphs. 

Theorem 4. For every r € Z + ,A > 1, and for every one of the six models described in the previous 
section, there exists (model dependent) z(r) such that w.h.p, we have 

lim N- 1 log Z(G(N,r)) = z(r) . (5) 

N-l-oo 
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4 Proofs: Optimization problems in Erdos-Renyi graphs 



The following simple observation will be useful throughout the paper. Given two hypergraphs Gj = 
([N], Ei), i = 1, 2 on the same set of nodes [N] for each one of the six models in Section [2] 

\H(G 1 )-H(G 2 )\=L\E 1 AE 2 \, (6) 

where we can take L = 1 for all the models except Ising, and we can take L = (3 for the Ising model. 
This follows from the fact that adding (deleting) an edge to (from) a graph changes the value of H by 
at most 1 for all models except for the Ising model, where the constant is j3. 
Our main technical result leading to the proof of Theorem [1] is as follows. 

Theorem 5. For every 1 < JVi, N 2 < N — 1 such that N\ + N 2 = N, and all models 

E[H(G{N, [cN\))\ > E[H(G(N 1 ,M 1 ))]+E[H(G(N 2 ,M 2 ))}, (7) 

where M 1 = Bi([cN\ 1 N 1 /N) and M 2 = [cN\ - Mi = BiflciVJ , N x /N). 

Additionally, for the same choice of Mj is as above and for Coloring, K-SAT and NAE-K-SAT 
models 

p(N,[cN\) >F(H(G(N 1 ,M 1 )(BG(N 2 ,M 2 ))=M 1 +M 2 ), (8) 

G\ © G2 denotes a disjoint union of graphs Gi,G2- 

Remark: The event H (G(A r i , M\) © 6(^2,^2)) = M\ + M 2 considered above corresponds to 
the event that both random graphs are satisfiable (colorable) instances. The randomness of choices of 
edges within each graph is assumed to be independent but the number of edges M. j are dependent since 
they sum to [ C ^J • Because of this coupling, it is not the case that 

F(H(G(Ni,Mi)®G(N 2 ,M 2 )) = Mi + M 2 ) =W{H{G{N 1 , M 1 )) = Mi)¥ (H(G(N 2 ,M 2 )) = M 2 ) . 

Let us first show that Theorem [5] implies Theorem [TJ 

Proof of TheoremUl Since Mj have binomial distribution, we have E[|.M,- — |_ C -^Y?JI] = 0(\/~N). This 
together with observation ([6]) and Theorem [5] implies 

E[H(G(N, [cN\))\ > E[H(G(N U [cN^))] + E[H{G(N 2 , [cN 2 \))] - 0(Vn). 

Namely the sequence E[H(G(N, [cN\))] is "nearly" super-additive, short of the 0(y/~N) correction 
term. Now we use Proposition [5] in the Appendix for the case a = 1/2 to conclude that the limit 
limAr^oo N^ 1 E[H(G(N, [cN\))] = H[c) exists. Showing that this also implies convergence of H(G(N, [cN\))/N 
to H(c) w.h.p. can be done using standard concentration results [JLROO] and we skip the details. It 
remains to show that H(c) is a non-decreasing continuous function for MAX-CUT, Coloring, K-SAT 
and NAE-K-SAT problems and is non-increasing for the Independent set problem. For the MAX-CUT, 
Coloring, K-SAT and NAE-K-SATall problems we have 

E[H(G(N, Mi))] <E[H(G(N,M 2 ))] 

when Mi < M 2 - adding hyperedges can only increase the objective value since the edge potentials are 
non-negative. For the Independent set problem on the contrary 

E[H(G(N,Mi))] >E[H(G(N,M 2 ))] 



S 



holds. The Lipschitz continuity follows from © which implies 

\E[H(G(N, Mi))] -E[H(G(N,M 2 ))]\ = L\M 1 -M 2 \, 
with L = f3 for for the Ising model, and L = 1 for the remaining models. This concludes the proof of 

We now turn to the proof of (|2|) and use ([8]) for this goal. Our main goal is establishing the following 
superadditivity property 

Proposition 1. There exist < a < 1 such that for all Ni, N 2 such that N = N\ + N 2 

logp(N, [cN\) > logp(iVi, LciViJ) + logp(iV 2 , LciV 2 J) - 0{N a ) (9) 

Part ([2]) of Theorem [1] then follows from this proposition and Proposition [5] from the Appendix B. 
The proof of Proposition Q] is found in Appendix A. □ 

We now turn to the proof of Theorem [5] and, in particular, introduce the interpolation construction. 

Proof of Theorem We begin by constructing a sequence of graphs interpolating between G(N, [cN\ ) 
and a disjoint union of G(N 1; Mi) and G(N 2 , [cN\ - Mi). Given N, N 1: N 2 s.t. JVi + N 2 = N and any 
< r < |_ciVj , let G(N, [cN\ , r) be the random graph on nodes [N] obtained as follows. It contains 
precisely [cN\ hyperedges. The first r hyperedges ei,...,e r are selected u.a.r. from all the possible 
directed hyperedges (namely they are generated as hyperedges of G(N, [cN\). The remaining |_ciVj — r 
hyperedges e r+ i, . . . , ei c jvj are generated as follows. For each j = r+1, . . . , [cN\ , with probability N\/N, 
ej is generated independently u.a.r. from all the possible hyperedges on nodes [N\], and with probability 
N 2 /N, it is generated u.a.r. from all the possible hyperedges on nodes [-/V 2 ](= {N\ + 1, . . . , N}). 
The choice of node and edge potentials H v ,H e is done exactly according to the corresponding model, 
as for the case of graphs G(N, [cN\). Observe that when r = [cN\, G(N, [cN\,r) = G(N, [cN\), 
and when r = 0, G(N, [cN\,r) is a disjoint union of graphs G(Nx,M.i),G(N 2 ,M. 2 ), conditioned on 
Mi+M 2 = [cN\, where Mj = Bi([cN\ , Nj/N). 
Proposition 2. For every r = 1, . . . , [ C -^J; 

E[H(G(N, [cN\,r))} > E[H(G(N, [cN\,r - 1))] 

Also for Coloring, K-SAT and NAE-K-SAT models 

f(h(G(N, [cN\,r)) = [cN\ \ > FUl(G(N, [cN\,r - 1)) = [cN\^j 

Let us first show how Theorem [5] follows from this proposition. Observe that for a disjoint union 
of two deterministic graphs G = G\ + G 2 , with G = (V, E),G\ = (Vi, E±),G 2 = (V 2 ,E 2 ), we always 
have H(G) = H(Gi) + H(G 2 ). The claim ([7]) then follows. The claim ([8]) follows immediately from the 
interpolation construction by comparing the cases r = and r = \ cN\ . □ 

Proof of Proposition® Observe that G(N, [cN\,r — 1) is obtained from G(N, [cN\,r) by deleting a 
hyperedge chosen u.a.r. independently from r hyperedges e 1; . . . , e r and adding a hyperedge either to 
nodes [N±] or to [N 2 ] with probabilities Ni/N and N 2 /N respectively. Let Go be the graph obtained 
after deleting but before adding a hyperedge. For the case of K-SAT and NAE-K-SAT (two models with 
random edge potentials), assume that Go also encodes the underlying edge potentials of the instance. 
For the case of Coloring, K-SAT, NAE-K-SAT, note that the maximum value that H can achieve for 
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the graph Go is [cN\ — 1 since exactly one hyperedge was deleted. We will establish a stronger result: 
conditional on any realization of the graph Go (and random potentials), we claim that 

E[H(G(N, [cN\,r))\G ] > E[H(G(N, [cN\,r - l))\G }. (10) 

And 

F(H(G(N, [cN\ , r)) = LciVj |G ) > F(H(G(N, [cN\ ,r-l))= [cN\ |G ) (11) 

for Coloring, K-SAT, NAE-K-SAT. Proposition then follows immediately from these claims by averaging 
over Go- Observe that conditional on any realization Go, G(N, [cN\,r) is obtained from Go by adding 
a hyperedge to [N] u.a.r. That is the generation of this hyperedge is independent from the randomness 
of Go- Similarly, conditional on any realization Go, G(N, [cN\,r — 1) is obtained from Go by adding a 
hyperedge to [Ni] or [N 2 ] u.a.r. with probabilities N\/N and N 2 /N respectively. 
We now prove properties (|10p and (|lip for each of the six models. 

Independent sets 

Let O* C [N] be the set of nodes which belong to every largest independent set in Go- Namely if 
/ C [N] is an i.s. such that \I\ = H(Gq), then O* C /. We note that O* can be empty. Then for 
every edge e = (i, k), H(G + e) = H(G ) - 1 if i, k G O* and #(G + e) = H(G ) if either i $ O* or 
k O* . Here Go + e denotes a graph obtained from Go by adding e. When the edge e is generated 



u.a.r. from the all possible edges, we then obtain E[iJ(Go + e)|Go] — H{Gq) = — y— — ) . Therefore, 
E[H(G(N, [cN\,r)\G ] - H(G ) = - (^V- By a similar argument 



lO* 



2 



E[H(G(N, [cN\,r - 1)|G ] - H{ 



Ni f\o*n [iVi]|V N 2 f\0* n[N 2 }\ 



N V Nx J N V ^2 

. _ 'iVi |o*n[iVi]| iv 2 |o* n [n 2 }\ 



N Ni N N 2 

E[H(G{N, [cN\,r)\G ] - H( 



N 



and (fTUI) is established. 



MAX-CUT 

Given Go, let C* C {0,1}^' be the set of optimal solutions. Namely H(x) = H(Go),\/x £ C* and 
H(x) < H(Go) otherwise. Introduce an equivalency relationship ~ on [N]. Given i,k G [N], define 
i ~ k if /or every x G C*,Xj = x^. Namely, in every optimal cut, nodes i and k have the same value. 
Let O* C [iV],l < j < J be the corresponding equivalency classes. Given any edge e = (i,k), observe 
that H(G + e) = #(G ) if i ~ A; and ff(G + e) = ff(G ) + 1 otherwise. Thus 
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Ising 

The proof is similar to the MAX-CUT problem but is more involved due to the presence of the magnetic 
field B. The presence of the field means that we can no longer say that H{G+e) = H(G)+(3 or = H(G). 
This issue is addressed by looking at suboptimal solutions and the implied sequence of equivalence 
classes. Thus let us define a sequence Hq > Hi > H2 ■ ■ ■ > Hm and an integer M > as follows. Define 
Hq = H(G). Assuming H m _\ is defined, m > 1, let H m = maxH(x) over all solutions x G {0,1}^ 
such that H{x) < H m -\. Namely, H m is the next best solution after H m -\. Define M to be the largest 
m such that H m > H(G) — 2/3. If this is not the case for all m, then we define M < 2 N to be the 
total number of possible values H{x) (although typically the value of M will be much smaller). Let 
C m = {x : H(x) = H m }, < m < M be the set of solutions achieving value H m . Observe that C m 
are disjoint sets. For every m < M define an equivalency relationship as follows i ~ k for i, k G [N] 
iff Xi = Xfc for all x G Co U • • • U C m . Namely, nodes i and k are m-equivalent if they take equal values 
in every solution achieving value at least H m . Let 0* m be the corresponding equivalency classes for 
1 < j < J m . Note that the partition Oj m+l of the nodes [N] is a refinement of the partition Oj m . 

Lemma 1. Given an edge e= (i,k), the following holds 



H(G + e) 



f H(G) + 0, ifi^k; 

H m+ i + p, if i ~ k, but i k, for some m < M 
H(G) -13, ifi™k for all m < M. 



m m+l 

Proof. The case i ^ k is straightforward. Suppose i ~ k, but i ^ k for some m < M — 1. For every 

m+l 

x G U m /< m C m / we have for some m! < m, Hg, +e (x) = H m / — (3 < Hq — (3. Now since i k there exists 
x G C m +i such that X{ 7^ Xk, implying HiQ +e (x) = H m+ \ + (3 > Hq — f3, where the inequality follows 
since m + 1 < M. Furthermore, for every x U m f < m C m f we have H<Q^ e (^x^j ^ Hq(x^) -\- (3 ^ H m ^\ -\- f3. 
We conclude that + /3 is the optimal solution in this case. 

On the other hand, if i ~ k for all m < M, then for all x G U m <MC m , H&+ e (x) < H(G) — f3, with 
equality achieved for x G Cq. For all x ^ U m <MC m , we have H(& +e (x) < Hm+i + /3 < H q — (3 and the 
assertion is established. Note that if M = 2 N , namely M + 1 is not defined, then U m <MC m is the entire 
space of solutions {0, 1}^, and the second part of the previous sentence is irrelevant. □ 

We now return to the proof of the proposition. Recall that if an edge e = (i, k) is added uniformly at 

(|o* 1 \ 
jy" J . A similar assertion holds for the case e is added uniformly 

at random to parts [Ni], I = 1,2 with probabilities Ni/N, respectively. We obtain that 



P(! ^) = i- E C¥) 

l<j<Jo V 7 

(is k , but i t fc) = y: C^r) 2 - E 



l<i<^m l<i<^m+l 
2 



AT 



(i ~ /c, Vm < M) = 

i<j<J M 



N 
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Applying Lemma [T] we obtain 

E[H(G(N, [cN\,r))\G } = (H + (3) I 1 



0<m<M-l 



1<3<JM 




0<m<M-l 



H + /3+ Yl 

+ (H~H M _ rj Xj x N 
By a similar argument and again using Lemma [T] we obtain 

E[H(G(N,[cN\,r-l))\G ] = H + p+ £ (H m+1 -H m ) £ E f ( ' ^ ^ ' 

0<m<M-l l<i<Jm 2=1,2 ^ ' 



1<J< Jm 2=1,2 



Recall, however that H m+ i — H m < 0, m < M — 1 and iJ — fljif — 2/3 < 0. Again using the convexity 
of the g(x) = x 2 function, we obtain the claim. 

Coloring 

Let C* C [q~] N be the set of optimal colorings. Namely H(x) = H(Gq),Vx G C*. Introduce an 
equivalency relationship ~ on the set of nodes as follows. Given i,k £ [N], define i ~ k iff X{ = x^ for 
every x E C* . Namely, in every optimal coloring assignments, i and k receive the same color. Then for 
every edge e, H(Gq + e) = H(Gq) if i ~ k and H(Gq + e) = H(Go) + 1 otherwise. The remainder of 
the proof of (|10p is similar to the one for MAX-CUT. 

Now let us show (jlip . We fix graph Go- Notice that if Go is not colorable then both probabilities 
in (jlip are zero, since adding edges cannot turn an uncolorable instance into the colorable one. Thus 
assume Go is a colorable graph. Since it has [cN\ ~~ 1 edges it means H(Go) = [cN\ — 1. Let 
O* C [N], 1 < j < J denote the ~ equivalence classes, defined by i ~ k iff in every proper coloring 
assignment i and k receive the same color. We obtain that 



H(G(N, [cN\,r)) = [cN\ 



1<3<J 



Similarly, 



F[H(G(N, [cN\,r - l)) = [cN\ 



Af ^ \ JVi / iV^V iV 9 J ' 



l<i<J 1<?'<J 
The relation (jlip then again follows from convexity. 
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K-SAT 



Let C* C {0, 1}^ be the set of optimal assignments. Define a node i (variable Xj) to be frozen if 
either Xj = 0,Vx G C* or Xj = l,Vx E C*. Namely, in every optimal assignment the value of i is 
always the same. Let O* be the set of frozen variables. Let e = . . . , ijc) C [N] be a hyperedge 
and let H e : {0,1}^ — > {0,1} be the corresponding edge potential. Namely, for some yi,...,yx G 
{0, 1}, H e (xi 1 , . . . , Xj fe ) = if Xi x = yi, ... , Xj K = uk and H e = 1 otherwise. Consider adding e with .£f e 
to the graph G . Note that if e D ([N] \ O*) ^ then F(G + e) = F(G ) + 1, as in this case at least 
one variable in e is non-frozen and can be adjusted to satisfy the clause. Otherwise, suppose e C O* , 
and let x* x , . . . , x* K G {0, 1} be the corresponding frozen values of i\, . . . , %k- Then H{Gq + e) = H(Gq) 
if x* =?/i,..., x* = Uk, and H(Gq + e) = H(Gq) + 1 otherwise. Moreover, for the random choice of 



H, the first event H(Gq + e) = H(Gq) occurs with probability 1/2 



A' 



E 



H(G(N, [cN\ ,r) G - H(G ) = 
and for every satisfiable instance Go (namely -ff (Go) = L C -^J 

F[H(G(N, [cN\,r)) = [cN\ 



We conclude that 



N 



1) 



1 



\0*\\K 



N 



Similarly, 

E[H(G(N, LciVj,r)|Go,^o] 
and for every satisfiable instance G| 
¥[H(G{N, [cN\,r)) = [cN\ 



H(G ) 



1 Ni 
1 JVi 

WW 



\o* n [iVi]| 



K 



|o*n[iVi]|\^ 



J_N2 

~ 2 K N 

J_N2 

2 K N 



\o* n [N 2 ]\\*< 

N 2 



\o*n[N 2 }\\ K 



Using the convexity of the function x K on x G [0, oo), we obtain the result 



NAE-K-SAT 

The idea of the proof is similar, and is based on the combination of the notions of frozen variables and 
equivalency classes. Two nodes (variables) i and k are defined to be equivalent % ~ k if there does not 
exist two assignments x, x' such that Xj = x' { , but X& ^ x' k , or vice verse: x» ^ x[, but x& = x' k . Namely, 
either both nodes are frozen or setting one of them determines the value for the other in every optimal 
assignment. Let Oj, 1 < j < J be the set of equivalence classes (the set of frozen variables is one of 
O*). Let e = C [N] be a hyperedge added to G , and let H e : {0,1}^ — > {0,1} be the 

corresponding edge potential. We claim that if . . . , %k are not all equivalent, then H(G + e) = H(G). 
Indeed, suppose without the loss of generality that i\ <ft i 2 and x, x' are two optimal solutions such that 
Xjj = x'j , Xj 2 x',- l2 . From the definition of NAE-K-SAT model it follows that at least one of the two 
solutions x and x' satisfies H e as well, and the claim then follows. Thus, H(G + e) = H{G) only if 
. . ,ik all belong to the same equivalence class. Provided that this indeed occurs, it is easy to see 
that the probability that H(G + e) = H(G) is 2/2 K . The remainder of the proof is similar to the one 
for the K-SAT model. 

We have established (jlOp and (jlip . With this, the proof of Proposition [2] is complete. □ 

Finally we give a simple proof of Corollary [TJ 

Proof of Corollary Ul Define c* H = sup{c > : H(c) = c}. It suffices to show that H(c) < c for all 
c > c* H . For every 5 > we can find cq G (c, c + 5) such that H(cq) < cq. By Lipshitz continuity result of 
Theorem [T]it follows that H(c) < H(cq) + {c — cq)<c for all c > cq and the assertion is established. □ 
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5 Proofs: Log-partition function in Erdos-Renyi graphs 

The following property serves as an analogue of ©. Given two hypergraphs Gj = ([N],E{), i = 1, 2 on 
the same set of nodes [N] for each one of the six models and each finite A 

| log Z(Gi) - log Z(G 2 )\ = 0(\E 1 AE 2 \). (12) 

This follows from the fact that adding (deleting) a hyperedge to (from) a graph results in multiplying 
or dividing the partition function by at most A for all models except for the Ising and Independent 
set models. For the Ising model the corresponding value is A^. To obtain a similar estimate for the 
independent set, note that given a graph G and an edge e = (u, v) which is not in G, we have 

z(G) = £Ai'i + £Ai'i 

eCl e<£I 

where in both sums we only sum over independent sets of G. We claim that 

J>i 7 i<A5>i'i 

eCl e(£I 

Indeed, for every independent set in G containing e = (u,v), delete node u. We obtain a one-to-one 
mapping immediately leading to the inequality. Finally, we obtain 

Z(G) > Z(G + e) = A |71 > — !-^Z(G), 

where our claim was used in the second inequality. The assertion f)12|) then follows after taking loga- 
rithms. 

The analogue of Theorem [5] is the following result. 
Theorem 6. For every 1 < Nx, N 2 < N — 1 such that Ni + N 2 = N and every A > 1 

E[logZ(G(Af, LeiVJ))] > E[log Z(G(N 1: Mi))] + E[log Z(G(N 2 , M 2 ))] 

where M 1 = Bi{[cN\,N 1 /N) and M 2 = [cN\ - Mi = Bi([cN\ , N x /N). 

As before, we do not have independence of Mj,j = 1,2. Let us first show how this result implies 
Theorem [2l 

Proof of Theorem [3 Since M j have binomial distribution, using observation (fT2l) and Theorem [6] we 
obtain 

E[logZ(G(iV, LcA^J))] > EpogZ(G(JVi, |cNiJ))] + E[Z(G(N 2 , [cN 2 \))] - 0(y/N). 
Now we use Proposition O in the Appendix for the case a = 1/2 to conclude that the limit 

lim Af~ 1 E[logZ(G(iV, LcA^J))] = z(c) 

W-s-oo 

exists. Showing that this also implies the convergence of A^ _1 E[log Z(G(N, [cN\))] to z{c) w.h.p. again 
is done using standard concentration results jJLROOj by applying property (fT2|) and we skip the details. 
The proof of continuity and monotonicity of z{c) for relevant models is similar to the one of H(c). □ 

Thus it remains to prove Theorem [6l 
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Proof of Theorem® We construct an interpolating sequence of graphs G(N, [cN\,r),0 < r < \cN\ 
exactly as in the previous subsection. We now establish the following analogue of Proposition [2j □ 

Proposition 3. For every r = 1, . . . , |_ciVj , 

E[logZ(G(iV, [cN\,r))} > EpogZ(G(JV, [cN\,r - 1))]. (13) 

Let us first show how Theorem [6] follows from this proposition. Observe that for disjoint union of 
two graphs G = Gi + G2, with G = (V, E),G\ = (Vi, E±),G2 = (^2,-^2), we always have logZ(G) = 
logZ(Gi) + logZ(G2). Theorem [6] then follows from Proposition [3l 

Proof of Proposition® Recall that G(N, [cN\,r — 1) is obtained from G(N, \_cN\,r) by deleting a 
hyperedge chosen u.a.r. independently from r hyperedges ei, . . . , e r and adding a hyperedge e either to 
nodes [N\] or to nodes [N2] with probabilities Ni/N and N2/N, respectively. Let as before Go be the 
graph obtained after deleting but before adding a hyperedge, and let Zq = Zo(Go) and fj,Q = /io,G De 
the corresponding partition function and the Gibbs measure, respectively. In the case of K-SAT and 
NAE-K-SAT models we assume that Go encodes the realizations of the random potentials as well. We 
now show that conditional on any realization of the graph Go 

E[logZ(G(iV,Lc7Vj,r))|Go] >E[logZ(G(iV,LciVj,r-l))|Go]. (14) 

The proof of (fT4"|) is done on a case by case basis and it is very similar to the proof of (fTOj) . 



Independent sets 

We have 

E[logZ(G(AT, [cN\ , r))|Go] - log Z = E[log g(gy^MJ 1 r )) | Gq ] 

Zo 

= E[log^ 

= E[log(l-//o(eC/ ))|Go], 

where the sums Y2i are over independent sets only and Iq denotes an independent set chosen randomly 
according to fi^. Notice, that since we are conditioning on graph Go the only randomness underlying 
the expectation operator is the randomness of the hyperedge e and the randomness of set Jo. Note that 
/io(e C Iq) < 1 since no(e <£_ Iq) > /Uq(/o = 0) > 0. Using the expansion log(l — x) = — J2m>i x m /m 



E[logZ(G(iV, LciVj,r))|G ] -logZo = -E[^ ^= ' 



° A G ] 



k=l 

00 



n E w^}— zs - |Go] 

k=l J 1 ,...,/* 

00 1 A^=i |/J| 
Efc E -^A— E[l {cC nJ =1 ^}|Go] 

k=l I 1 ,...,I k 

- 1 x^UW f\n*P\\ 2 



E h E 



z k I N 
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where the sum Ei 1 jk is again over independent subsets I 1 ,. . . ,I k of Go only, and in the last equality 
we have used the fact that e is distributed u.a.r. Similar calculation for log Z(G(N, \ cN\ , r — 1)) that is 
obtained by adding a hyperedge to nodes [Aq] with probability Ni/N, or to nodes [N 2 ] with probability 
N 2 /N, gives 

E[ log Z(G(N, [cN\ , r - 1))|G ] - log Z 

Aq ( I n} =1 P n[N!]\Y n 2 (\ r\) =1 P n [n 2 ] \ \ 



~ 1 Ag^l 
^ k ^ zk 

k=l I 1 ,...,/* 



AM JVi I N \ N 2 I 

Again using the convexity of /(x) = x 2 we obtain 

E [log Z(G(N, [cN\ , r))|Go] - log Z >E [log Z(G(N, [cN\ , r - 1))|G ] - log Z 
and (ITU) is established. 



MAX-CUT. Similarly to the Independent set model, if G(N, [cN\ , r) is obtained from Go by 
adding an edge (i, j) where i,j are chosen uniformly at random, we have 

E[logZ(G(A, [cN\,r))\Go] - logZ = E[log Z{G{N > [dVj ' r)) |G ] 

= E[log E^A^ 

= log A + E[log(l - (1 - -)no(xi = Xj-))|Go]. 
Since A > 1 we have < (1 — A _1 )/io(^j = £j) < 1 (this is where the condition A > 1 is used) implying 

E[logZ(G(A, [cN\,r))\G ] - logZ - log A = -E[f] (1 ~ = a ^ |G ] 

k=l 



OC 



^ k 1 ^ Zk ^ x i- x j 

k=l xi,...,x k U 

y (1 - A- X ) fc ^ A^^) 

= -1^ — i — 1. — Y k — E[1 K=4' 

fc=l a;i,...,Xfc 

Now for every sequence of vectors x±,...,Xk introduce equivalency classes on [N]. Given i,k £ [N], say 
i ~ k if x\ = x k , W = 1, . . . , k. Namely, in every one of the cuts defined by xi, I = 1, . . . , k, the nodes i 
and k belong to the same side of the cut. Let O s , 1 < s < J be the corresponding equivalency classes. 

For an edge e = generated u.a.r., observe that E ^l{ ar e =!C <v£}.|Gol = Es=i (^lv^) ■ Thus 



00 n _ \^\k xeLi^w j no iv 

E[logZ(G(iV,LciVj,r))|Go]-logZo-logA = -^^ ±-L £ ~ ^ 

i 1 T, ^, e— 1 \ / 



fc=l Xi,...,X k U s= l 

and similarly, 
E[log Z(G{N, [cN\ , r - 1))|G ] - log Z - log A 

f (1-A-? v \T,UH(t) J( Nl no^mi y n 2 no s n[N 2 ]\ V 

k k xh, z ° k\ N ^ ^ / * v ^ ; 
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Using the convexity of the function f(x) = x 2 , we obtain (|14p . 

Ising, Coloring, K-SAT and NAE-K-SAT. The proofs of the remaining cases are obtained 
similarly and is omitted. The condition A > 1 is used to assert positivity of 1 — A" 1 in the logarithm 
expansion. □ 



6 Proofs: Random regular graphs 

For the proofs related to random regular graphs we will need to work with random "nearly" regular 
graphs. For this purpose, given N, r and K such that Nr/K is an integer and given any positive integer 
T < Nr/K, let G(N, r, T) denote the graph obtained by creating a size T matching on Nr nodes of the 
configuration model uniformly at random and then projecting. For example if T was Nr/K, then we 
would have obtained the random regular graph G(N,r). 

Our result leading to the proof of Theorems [3] is as follows. 

Theorem 7. For every N\,N 2 such that N = JVi + N 2 and N±r j 'K,N 2 r / K are integers, 

E[H(G(N,r))} > E[H{G(N 1 ,r))]+E[H(G{N 2 ,r))] - O(ivt). (15) 

Proof. Fix Ni,N 2 such that Ni +N 2 = N and N^r/K, N 2 r/K are integers. Let us first prove Theorem[7] 
for the simpler case mhx^i^ Nj < 40N&. In this case starting from the graph G(N,r) we can obtain 
a disjoint union of graphs G(Nj,r) via at most O(Ne) hyperedge deletion and addition operations. 
Indeed, suppose without the loss of generality that Ni < AON^ . Delete all the hyperedges inside [N-\] as 
well as all the hyperedges connecting two parts. Then generate a random graph G(N\,r) from scratch. 
Finally, complete a so obtained partial matching in the configuration model on [N 2 r] and project. The 
total number of deleted and generated hyperedges is O(Ne) and indeed we obtain a disjoint union of 
graphs G(Nj,r),j = 1,2. Since the hyperedge deletion and generation operation changes the value of 
H by at most 0(N%), then the proof of (|15p follows. 

Now, throughout the remainder of the section we assume minj = i 2 Nj > 40iV6. Fix T = Nr/K — 
[(l/K)N%\ and consider the graph G(N, r, T). Note that Nr/K -T = O(ivl). 

We now describe an interpolation procedure which interpolates between G(N, r, T) and a union of 
certain two graphs on nodes [Ni] and [N 2 ], each of which will be "nearly" regular. For every integer 
partition K = K\ + K 2 such that K\,K 2 > 1 let Tk 1: k 2 < J 1 be the (random) number of hyperedges 
which connect parts [N\] and [N 2 ] in G(N, r, T) and such that each connecting hyperedge has exactly 
Kj nodes in part [Njr] in the configuration model. Let To = YIki K2>i k 1 +k 2 =kTk 1 ,k 2 - Observe that 
T < mm j=1 ^ 2 (Njr). 

Define G(N,T,0) = G(N,r,T) and define G{N,T,t), l<t< T x . K -\ recursively as follows. Assum- 
ing G{N , T,t — 1) is already defined, consider the graph Go obtained from G(N, T, t — 1) by deleting a 
hyperedge connecting [iVi] and [N 2 ] chosen uniformly at random from the collection of hyperedges which 
have exactly 1 node in part [N\r] and K — 1 nodes in part [N 2 r] (from the remaining T\.k-\ — (t — 1) 
such hyperedges). Then we construct G(N, T, t) by adding a hyperedge to the resulting graph as follows: 
with probability 1/K a hyperedge is added to connect K isolated nodes chosen uniformly at random 
among the isolated nodes from the set [iVir]. With the remaining probability {K — 1)/K a hyperedge 
is added to connect K isolated nodes chosen uniformly at random among the isolated nodes from the 
set [A^r]. It is possible that at some point there are no K isolated nodes available in [Njr]. In this case 
we say that the interpolation procedure fails. In fact we say that the interpolation procedure fails if in 
either of the two parts the number of isolated nodes is strictly less than K, even if the attempt was 
made to add a hyperedge to a part where there is no shortage of such nodes. 
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Thus we have defined an interpolation procedure for t < T^k-i- Assuming that the procedure did 
not fail for t < T^k-i, we now define it for T\ y K-i + 1 < t < T2 t K-2 analogously : we delete a randomly 
chosen hyperedge connecting two parts such that the hyperedge has 2 nodes in part j = 1, and K — 2 
nodes in part j = 2. Then we add a hyperedge uniformly at random to part j = 1,2 to connect K 
isolated nodes with probability 2/K and (K — 2)/K respectively. The failure of the interpolation is 
defined similarly as above. We continue this for all partitions {K\,K2) until (K — 1, 1), inclusive. For 
the (K\,K2) phase of the interpolation procedure the probabilities are K\/K and K2/K respectively. 

Let It be the event that the interpolation succeeds for the first t steps, and let I = C\t<T It denote 
the event that the interpolation procedure succeeds for all steps. For simplicity, even if the interpolation 
procedure fails in some step t' we still define G(N,T,t),t' < t < Tq to be the same graph as the first 
graph at which the interpolation procedure fails: G(N,T,t) = G(N,T,t'). It will be also convenient to 
define G(N,T,t) = G(N,T,Tq) for Tq < t < min^i^A^r), whether the interpolation procedure fails 
or not. This is done in order to avoid dealing with graphs observed at a random (To) time, as opposed 
to the deterministic time min,-=i 2(iV,-r). 

Provided that the interpolation procedure succeeds, the graph G(N, T, min^i^ Njr) is a disjoint 
union of two graphs on [Nj],j = 1,2 each "close" to being an r-regular random graph, in some appro- 
priate sense to be made precise later. 

Our next goal is establishing the following analogue of Proposition [2l As in previous sections, let 
Go denote the graph obtained from G(iV, T,t — 1) after deleting a hyperedge connecting two parts, but 
before a hyperedge is added to one of the parts, namely, before creating G(N,T,t), conditioned on 
t < Tq and the event that the interpolation process succeeds till t - r\t'<tltf- If> on the other hand the 
interpolation procedure fails before t, let Go be the graph obtained at the last successful interpolation 
step after the last hyperedge deletion. Let Aj denotes the degree of the node i E [N] in the graph Go 
and let 

denote the number of isolated node in the j-th part of the configuration model for Go for j = 1,2. 
Proposition 4. For every t < min^ Njr 

E[H(G(N,T,t-l))}>E[H(G(N,T,t))]-0(Em&x^y (16) 

Proof. The claim is trivial when Tq + 1 < t, since the graph remains the same. Notice also that 

E[H(G(N,T,t- l))|XtJ = E[if(G(iV,T,t))|Xti] , 

since the two graphs are identical, and thus the statement of the proposition holds. 
Now we will condition on the event If We now establish a stronger result. Namely, 

E[H(G(N, T, t - 1))|G ] > E[H(G(N, T, t))\G ] - o( max — L^) . (17) 

Observe that conditioned on obtaining graph Go, the graph G(N,T,t — 1) can be recovered from Go 
in distributional sense by adding a hyperedge connecting K\ isolated nodes from [N%r] to K2 isolated 
nodes from [iV^r], all chosen uniformly at random, and then projecting. 
We now conduct model dependent case by case analysis. 

Independent sets. In this case K = 2 and the only possibility is K\ = K2 = 1. As in the previous 
section, O* again denote the set of nodes in [N] which belong to every largest independent set in Gq. 
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Then in the case of creating graph G(N, T, t — 1) from Go, the newly added edge e decreases H by one 
if both ends of e belong to O*, and leaves it the same otherwise. The first event occurs with probability 



Ei 1 eo*n[7Vi],i 2 eO*n[7V 2 ] ( r A h)v ^a) _ Ei 



eO*n[iVi 



ieo*n[N 2 



(r- Aj 



e[iVi],i 2 e[JV : 



\(r - AjJ(r - Aj 



Eie[7Vi]( r ~~ A«) E 



ie[Ar 2 ] 



Aj) 



We now analyze the case of creating G(N,T,t). Conditioning on the event that e was added to part 
[Njr], the value of H decreases by one iff both ends of e fall into O* n [Nj]. This occurs with probability 



E, 



£[0*nNj 



E, 



GO*n[Ay l r 



Aj 



E 



i£[0*nN. 



A, 



E, 



Therefore, the value of H decreases by one with probability 

2 



Eie[Ar,l( r _ A 



o 



-T. 

3=1,2 



E 



A,- 



0\ max 



3=1,2 Zj(t) J' 



and stays the same with the remaining probability. Using the inequality (1/2) (x 2 + y 2 ) > xy we obtain 

dm). 

MAX-CUT, Ising, Coloring. As in the proof of Theorem [T] we introduce equivalence classes 
Oj C [iV],l < j < J on the graph Go- The rest of the proof is almost identical to the one for the 
Independent Set model and we skip the details. Notice, that in all of these cases we have K = 2 and 
the interpolation phase has only one stage corresponding to (Ki, K 2 ) = (1, 1). 

K-SAT. This is the first model for which K > 2. We fix K x , K 2 > 1 such that K x + K 2 = K and 
further condition on the event that the graph Go was created in stage (Ki,K 2 ). As in the previous 
section, let O* be the set of frozen variables in all optimal assignments of Go- Reasoning as in the 
previous section, when we reconstruct graph G(N, T, t — 1) in the distributional sense by adding a 
random hyperedge connecting K\ nodes in [N\r] with K 2 nodes in [-/V^r], the probability that the value 
of H remains the same (does not increase by one) is precisely 

1 [ Ejgo*n [Af 1 ]( r ~ Ai)] A 'i |-EieO*n[Ay ^ r ~~ Ai 
2 K 



J2ie[N! 

Similarly, creating G(N, T, t) from Go 



Aj 



K 2 



J_Ki 
2 K K 



E, 



eO*n[JVi 



Aj 



K 1 K2 

+ 2 K K 



Eie[AT 2 ]\- — - 

keeps the value of H the same with probability 
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EiGfAfol( r ~ A i 



i£[N 2 
i 



O ( max =— 

j =1 > 2 Eie[jv^]( r 



A, 



(19) 



Applying Young's inequality, namely that ab < pap + qbi for every a,b > 0,p + q = l,p,q > 0, with 
the choice p = K\/K,q = K 2 /K, 

rEieO*n[iVi] ( r — A i 
-Aj) 



Eie^!] 1 
Ejgo*n[Af 2 ]( r ~ A ») i K - 
Eie[AT 2 ]( r ~ A i) - 



and canceling 1/2 on both sides, we obtain the result. 

NAE-K-SAT. The proof is similar to the one for K-SAT and for NAE-K-SAT for the G(N, [cN\) 
model. This completes the proof of the proposition. □ 
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Our next step is to control the error term in (|16p . 

Lemma 2. The interpolation procedure succeeds (eventX holds) with probability at least 1— 0(A^exp(— N s )) 
for some 5 > 0. Additionally, 



1<*<T JK ' 

Proof. Since Go is obtained after deleting one hyperedge connecting two parts, but before adding a 
new hyperedge, then Zj{t) > 1. A crude bound on the required expression is then E[To] = O(minA^). 

2 11 

We have E[Z 3 -(0)] = (Nj/N)Ns = Nj/Ns > 40N? since the initial number of isolated nodes was 

2 5 

Nr/K — T = N3 and min,,- Nj > 40N&. Moreover, using a crude concentration bound ¥(Zj(0) < 
(l/2)(Nj/N*) + K) = 0(exp(-A^)) for some S\ > 0. Observe that Zj(t + 1) - Zj(t) = with 
probability one if the interpolation procedure failed for some t' < t. Otherwise, if t corresponds to 
phase (Ki,K^) then Zj(t + 1) — Zj(t) takes values —Kj + K with probability Kj/K and —Kj with 
the remaining probability. This is because during the hyperedge deletion step Zj(t) decreases by Kj 
and during the hyperedge addition step it increases by K or by zero with probabilities Kj/K and 
1 — Kj/K respectively. In particular, K[Zj(t + 1) — Zj(t)\ = 0. The decision of whether to put the 
hyperedge into part 1 or 2 are done independently. Since t < Tq < Nj, we conclude that for each 
t < T we have P(Zj(0) - Zj(t) > A^ 3/5 ) = 0(exp(-A^ 2 )) for some 5 2 > 0. Here any choice of 
exponent strictly larger than 1/2 applies, but for our purposes 3/5 suffices. It follows that, Zj(t) > 
(l/2)Nj/N* + K - Nf b for all t with probability 1 - 0(Nj exp(-A^)) = 1 - 0(Nexp(-N 5 )) for 
5 = min(5i, 62)- The assumption minNj > 4CW6 implies that a weaker bound miniVj > 325 A^6 , which 
translates into (l/2)Nj/N* -N? /5 > 0. Thus Zj(t) > K for all t, with probability 1 - 0(N exp(-N 5 )), 
and therefore the interpolation procedure succeeds. 

Now ignoring term K in the expression {\/2)Nj/Ni-\-K—N-' and using To < mmj(Njr), we obtain 
that with probability 1 - 0(N exp(-A^ <5 )), the expression inside the expectation on the left hand-side 
of (f20l) is at most 

2 

N jr _ NJ-r 



(1/2)^-3 - Nf° (l/2)N]N 



3 - 1 



The numerator is at most N^r. Also the assumption mm.Nj > 40N& implies that the denominator 

2 

is at least 1. We conclude that the expression inside the expectation is at most Nsr with probability 
at least 1 — 0(N exp(— N s )). Since we also have To < Nr w.p.l, then using a very crude estimate 
0(N ex.p(-N 5 )) = 0(N~i), and NN~^ = Jvf , we obtain the required result. □ 

As a corollary of Proposition 2] and Lemma [2] we obtain 
Corollary 2. 

E[H(G(N,T,0))\ > K[H(G(N,T, min Njr))] -0(JV§). 

j=l,2 

Let us consider graph G(N, T, Tq)). We further modify it by removing all hyperedges which connect 
two parts [Nj] of the graph, if there are any such hyperedges left. Notice that if the event X occurs, 
namely the interpolation procedure succeeds, no further hyperedges need to be removed. The resulting 
graph is a disjoint union of graphs obtained on nodes [N±r] and [iV^r] by adding a random size partial 
matching uniformly at random. The actual size of these two matchings depends on in the initial size of 
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the partial matching within each part, and also on how many of To hyperedges go into each part during 
the interpolation steps, and how many were removed in the final part (if any). We now obtain bounds 
on the sizes of these matchings. 

Recall minj Nj > 40iV"6. We showed in the proof of Lemma [2] that the interpolation procedure 
succeeds with probability 0(N exp(— N s )) for some 5. This coupled with the fact that w.p.l, the 
number of hyperedges removed in the final stage is at most rN/K, gives us that the expected number 
of hyperedges removed in the final stage is at most 0(N 2 exp(— N s )) which (as a very crude estimate) 

2 2 

is O(iVs). Moreover, since the initial number of isolated nodes was Ns and during the interpolation 
procedure the total number of isolated nodes in the entire graph never increases, then the total number 

2 

of isolated nodes before the final removal of hyperedges in G(N, T, Tq) is at most Nz . We conclude that 

2 

the expected number of isolated nodes in the end of the interpolation procedure is O(Nz). Then we can 
complete uniform random partial matchings on [iV,-r] to full uniform random matchings by adding at 
most that many hyperedges in expectation. The objective value of H changes by at most that much as 
well. The same applies to G(N, r, T) - we can complete the configuration model corresponding to this 

2 2 

graph to a full matching on Nr nodes by adding at most Ns hyperedges since Nr/K — T = 0{Ns). 
Coupled with Corollary [2] we then obtain 

E[H(G(N,r))} > E[H(G(N u r))] + E[H(G(N 2 ,r))] - O(m), 

5 

for the case minj Nj > 40N~s . This completes the proof of Theorem [71 □ 
Proof of Theorem [31 The existence of the limit 

lim iV- 1 E[F(G(iV, r))l = H(r) 

N^oo,N£r- 1 KZ + 

follows immediately from Theorem [7J and Proposition [5] from the Appendix. Then the convergence 
w.h.p. 

lim N- l H(G(N, r)) = H(r) , 

N~^oo,N£r~ 1 K1 + 

follows once again using standard concentration results [JLR OO]. 

□ 

The proof of Theorem H] uses the same interpolation as the one above and the proof itself mimics 
the one for Theorem [2j For this reason, we omit the details. 
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Appendix A. Proof of Proposition [I] 

Proof of Proposition^ Fix any 1/2 < u < 1. First we assume N% < N u . Let Aij be as in Theorem [5j 
We have 

P{H (G(N u Mi) (BG{N 2 ,M 2 )) = Mi + M 2 ) >p(N 2 , [cAfeJMWi, [cN\ - [cN 2 \)¥{M 2 = [cN 2 \). 
We have 

P(M 2 = [cN 2 \) = (^^(Nz/lfj^iNt/N)^-^. 

From our assumption N\ < N u it follows that 

(N 2 /N)t cN ^ > (1 - A^-i)O(JV) = eX p(-0(A^)), 
(JVi/JV)L«^J-L^J > (i/AT)0(JVi) >exp(-0(iV iy logiV)). 

It then follows 

P(M 2 = [cN 2 \) > exp(-0(N u logN)). 

Now we claim the following crude bound for every every deterministic m and every one of the three 
models under the consideration. 

p(N, m + 1) > 0(l/N)p(N, m) 

Indeed, for the K-SAT model, conditional on the event that H(G(N,m)) = m, the probability that 
H(G(N,m + 1)) = m + 1 is at least 1 — 1/2 K . We obtain thus a bound which is even stronger than 
claimed 

p(N, m + 1) > (1 - l/2 K )p(N, m) = 0(p(N, m)). 

The proof for the NAE- K-SAT is similar. For the Coloring problem observe that this conditional 
probability is at least (1 - 1/N)(2(N - l)/N 2 ) = 0(1 /N) since with probability 1-1/N the new edge 
chooses different nodes, and with probability at least 2(N — 1)/N 2 the new edge does not violate a given 
coloring, (with equality achieved only when q = 2 and two coloring classes having cardinalities 1 and 
N — 1). The claim follows. 

Now since |_cA^J — [cA^J < L c -^iJ + 1) the claim implies 

p(N u [cN\ - [cN 2 \) > 0{l/N)p{N u LciViJ) 

Combining our estimates we obtain 

P(H{G(Ni,Mi)®G(N 2 ,M2)) = Mi+M 2 )>p{N l , [cNi\)p{N 2 , [cN 2 \)0(l/N)exp(-0(N u logN)). 

After taking logarithm of both sides we obtain from (JSj) . 

The case N 2 < N u is considered similarly. We now turn to a more difficult case Nj > N u , j = 1,2. 
We first establish the following claim for the three models of interest (Coloring, K-SAT, NAE- K-SAT). 
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Lemma 3. The following holds for Coloring, K-SAT, NAE-K-SAT models for all N,M,m and < 
5 < 1/2 

p(N, M + m)> 5 m p(N, M) - (25) M+1 exp(H(5)N + o(N)) (21) 
where H(S) = — <51og<5 — (1 — 5) log(l — 5) is the entropy function. 

Proof We first assume K-SAT or NAE-K-SAT models. Let us show for these models that there exists 
a constant u > 1/2 such that for every graph and potential realization (G = (V,E),H) such that the 
problem is satisfiable (namely H(G) = \E\), if a randomly chosen hyperedge e is added with a potential 
chosen according to the model, then 

F(H(G + e) = \E\ + 1) > uj. 

In other words, if the current graph is satisfiable, the new graph obtained by adding a random hyperedge 
remains satisfiable with probability at least uj. Indeed, for example for the case of K-SAT, if the instance 
is satisfiable and x is a satisfying assignment, the added edge remains consistent with x with probability 
at least u = 1 - l/2 K > 1/2. For the case of NAE-K-SAT it is u = 1 - 1/2*- 1 > 1/2. We obtain that 
for every positive M,m and recalling assumption 5 < 1/2, 

p{N, M + m)> u m p(N, M) > 5 m p{N, M), 

and the assertion is established. 

The proof for the case of Coloring is more involved. Given < 5 < 1/2 we call a graph GonN 
nodes <5-unusual if it is colorable and in every coloring assignment there exists a color class with size at 
least (1 — 5)N . Namely, for every color assignment x such that H(x) = \E\, there exists k £ [q~] such 
that the cardinality of the set {i £ [N] : X{ = k} is at least (1 — 5)N. We claim that 

P(G(N, M) is 5-unusual) < (25) M exp(H(5)N + o(N)). (22) 

The claim is shown using the first moment method - we will show that the expected number of graphs 
with such a property is at most the required quantity. Indeed, given a subset C C [N] such that 
|C| > (1 — S)N, the probability that the graph G(N, M) has proper coloring with all nodes in C having 
the same color is at most (1 — (1 — 5) 2 ) M < (2S) M , since we must have that no edge falls within the class 
C. There are at most (i^) = exp(H(5)N + o(N)) choices for the subset C. The claim then follows. 

Now observe that if a graph G = (V, E) is colorable but not <5-unusual, then adding a random edge 
e we obtain P(H(G + e) = \E\ + 1) > 5(1 — 5). Namely, in this case the probability goes down by at 
most a constant factor. We obtain 

p(N, M + 1) > P(H(G{N, M + 1) = M + 1)|G(JV, M) colorable & not 5-unusual) 
x F(G(N,M) colorable & not <5-unusual) 

> (5(1 - 5)P(G(N,M) colorable & not 5-unusual) 

> (5(1 - S)P(G(N,M) colorable) - 5(1 - 5)F(G(N,M) <5-unusual) 

> (5(1 - 5)p(N,M) - 5P(G{N,M) (5-unusual) 

> (5(1 - 5)p(N, M) - 5{25) M exp(H(5)N + o(N)), 

using the earlier established claim. Iterating this inequality we obtain for every m > 1 

p(N,M + m)>5 m (l-5) m p(N,M))-5(25) M eMH(5)N + o(N)) 5 rn (l-5) m 

0<j<m-l 

> 5 m P(H(G(N,M) = M)) - (25) M+1 exp(H(5)N + o(N)), 

where J2o<j<m^ m (^- ~ d) m < 1/(1 — (5) < 2 is used in the last inequality. This completes the proof of 
the lemma. □ 
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We now prove (j2J). Fix h G (1/2, z/). We have from flSJ) 

p(iV, LciVj) >P(i?(G(iVi,Mi)eG(iV 2 ,^ 2 )) =M 1 + M 2 ) 

J] p(iVi,miMiV 2 ,m 2 )P(Mi = mi). 

cNi -N h <mi <cNi +N h ,rri2 = [cN\ -mi 



> 



Note that cNi - N h < mi < cN x + N h implies cN 2 - N h - 1 < m 2 < cN 2 + N h . Applying Lemma [3] 
we further obtain for the relevant range of rrij that 

p{Nj,mj) > tfw-W'^piNj, [cNj\) - (25) cN i exp(H(S)Nj + (Nj)) 

> 5 Nh+1 p(N j7 [cNj\) - (26) cN i exp(H(S)Nj + o(Nj)) 

> S Nh+1 p(N„ [cNj]) (l - 2 ^8^- Nh -\l - l/q)-"* exp(H(S)Nj + o(JVj))) , 

where we have used a simple bound p(Nj, [cNj\) > (1 — l/q) cNj . Now let us take 5 so that 

(3(5) 4 (25(1 - l/g))~ c exp(tf(<5)) < 1. (23) 
Then using the assumption Nj > iV 1 ' and h < u we obtain 

p{N 3 ,m 3 ) > 5 Nh+1 p{N 3 , [cN 3 \) (l - (/?(5))°^) . 

Combining we obtain 

p{N,[cN\) > P(ciVi - N h < Mi < cNi + N h ) J| S°^%(Nj, [cNj\) (l - (P(5))° {Nn ^ 

3=1,2 

This implies 

logp(iV, LcA^J) > logP(ciVi - N h < Mx < cNx + N h ) 

+ N h \og5+ logpiN^lcN^ + log^l-tm) ^) 

3=1,2 

Since Mi = Bi([cN\,Ni/N), and h > 1/2, then logP(ciV"i - N h < Mi < cN x + N h ) = o(l). Since 
P(6) < 1 then 

log (i - mf {Nn ) = o (mf {Nn ) = o(n% 

where the last identity is of course a very crude estimate. Combining, we obtain 

Iogp(JV, LciVj) > l °SP(N 3 , [cNj]) + 0(N h ) 
i=i,2 

The claim of Proposition [1] is established. 

Appendix B. Modified super-additivity theorem 

Proposition 5. Given a G (0,1), suppose a non-negative sequence ajy,N > 1 satisfies 

ajsr > ajvi + cln 2 - 0(N a ). 
for every N\, N 2 s.t. N = N± + N 2 . Then the limit limjv-5.00 exists. 



□ 



(24) 
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Proof. It is convenient to define ajy = a[JVJ f° r every real not necessarily integer value N > 1. It is 
then straightforward to check that the property ([2"3|) holds when extended to reals as well (thanks to 
the correction term 0(N a )). Let 

* r a N 
a = limsup — -. 

JV-voo A 

Fix e > and find k such that 1/k < e < l/(k — 1). Find find A = N (e) such that N^a^ > a* - e, 
^a N a-\ < £> clearly, such Nq exists. Consider any iV > kN . Find r such that kN 2 r < N < kN 2 r+1 . 
Applying ([2"3]) iteratively with Ai = A2 = N/2 we obtain 

a N >2 r aN r - 0(2 l ' N ' 

0</<r-l 



2' 

^-o(2^- a >N°^, 



Now let us find i such that (fe + «)7Vo < N/2 r < (k + i + 1)Nq. Note i < k. Again using (f2l|) successively 
with A for A 7 ! and N/2 r , (N/2 r ) - N , (N/2 r ) - 2A , . . . for A 2 , we obtain 



aN r >(k + i)a No - 0[k(— 
>(k + i)a No -0(k(^r 



Combining, we obtain 



a N > 2 r {k + i)a No - 0[2^- a >N a ) - 0[k2 r ^~^N c 
= 2 r (k + i)a No -o(k2 r ^-°^N a \ 



Then 



on_ > 2 r (k + i) a No _ ( k2 r(i-a)N<x- 
N ~ 2 r {k + i + l) N \ 

> (1 - - -){a* -e)-0( fcfC^'r- 1 ) 

> (1 - e)(o* - e) -0{k2 r ^-^N a ^ , 

where 1/k < e is used in the last inequality. Now 

k2 r(i-a) N a-l < k2 r{1 - a \k2 r N ) a ~ l = k a N^ 1 < e, 
again by the choice of Nq. We have obtained 

^>(l- e )(a*- £ )- e , 

for all A > N$k. Since e was arbitrary the proof is complete. □ 
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